…nels

Re-running `benches/decimal_assemble.rs` after the review corrected a
claim the previous commit made. Specializing the part count is worth
1.85x on `i256`, as reported, but on `i128` it is worth only ~1.04x — the
1.25x figure did not reproduce. What actually costs on `i128` is the
write: pushing into a reserved buffer instead of storing into a pre-sized
one is the whole difference at 16 bytes per row.

A new `i128_row_write` variant isolates it, holding the loop shape fixed
and changing only the output buffer. Over 65,536 rows, `fastest` of three
runs each:

| shape                        | i128    | i256    |
| ---------------------------- | ------- | ------- |
| row, runtime part count      | 143 µs  | 351 µs  |
| row, const part count, push  | 138 µs  | 190 µs  |
| row, const part count, write | 83 µs   | 196 µs  |
| column, lane writes          | 103 µs  | 438 µs  |

So the columnar shape was never the interesting axis: it beats the
*pushing* row loop on `i128` but still loses to the single-pass write,
and the second pass buys nothing once the push is gone. On `i256` the
write shape ties the push shape, because 32 bytes of stores per row
dominate either way, so only `assemble_i128` changes. Through the array
API, one lower part goes 138 µs -> 83 µs (1.6x); three parts is unchanged
at ~209 µs.

The rest is cleanup from the same review.

Seven kernels open-coded "map every part, rebuild the array", and two of
them had already been fixed in this branch for dropping the lower parts
on the floor. `map_parts`, `with_msp` and `decimal_dtype` replace all
seven, so a part-wise op cannot silently lose a part, and the argument
for why an MSP-only rebuild is sound lives in one doc comment instead of
being restated or omitted per site.

Dead code: `DecimalBytePartsDataParts` had exactly one reference in the
repository — its own definition — and this branch had been growing it a
field and doc comments. The `[first]` arm of the `i256` dispatch is
unreachable, since one lower part under a <=64-bit MSP always lands in an
`i128`; a comment now says so where the arm was.

Visibility: `assemble_decimal`, `assembled_values_type` and
`LOWER_PART_DTYPE` had no callers outside the crate and are now
crate-private. `assemble_decimal` was public only so the benchmark could
call it, but `canonicalize_byte_parts` already measures the same assembly
through the array API, so the two `*_assemble_shipped` benches go with
it. As public API it could also panic rather than error on an unsigned
MSP, since signedness is only checked on the zero-parts path.

The metadata accessor `lower_parts()` returned a count while the
generated slots accessor of the same name returns the arrays, both in
scope in the same module; it is now `lower_part_count()`. The btrblocks
scheme spelled the child layout as `1 + MAX_LOWER_PARTS` and `idx + 1`
where the encoding crate has named slot constants; it now uses them.
Three hand-rolled LCGs become `StdRng::seed_from_u64`, matching the rest
of the repo. Four one-line rejection tests became one `rstest` in the
previous commit; the two removed columnar bench variants are recorded in
the module doc with their numbers rather than kept as dead code.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Auditing each compute function against the reduce/execute contract —
`*Reduce` operates "purely on array metadata and structure without
needing to read or execute on the underlying buffers", `*Kernel`/
`*Execute` may read buffers and take an `ExecutionCtx` — turned up two
kernels on the wrong side of it.

`take` was implemented as `TakeExecute` and registered as an execute
parent kernel, but its body ignores the context entirely: `ArrayRef::take`
wraps each part in a `Dict` and optimizes, which is a lazy rewrite, and
the only other work is the `validate` call rebuilding the array. It is
now `TakeReduce`, registered through `TakeReduceAdaptor` alongside the
other parent reduce rules, so the push-down happens during optimization
rather than being deferred to execution. `TakeReduceAdaptor` also applies
the empty-indices and empty-array preconditions and propagates take
statistics, neither of which the execute path was doing. The nullable
indices guard keeps its meaning: `Ok(None)` now means "cannot do this
without buffers", which is exactly the fallback it was asking for.

`DecimalBytePartsFilterPushDownRule` was byte-for-byte what
`FilterReduceAdaptor(DecimalByteParts)` already does via `FilterReduce`,
and was listed first so it shadowed the adaptor — which meant filtering
also skipped the adaptor's empty-mask preconditions. Removed; the adaptor
that was already registered covers it.

The other kernels are on the correct side and stay put. `filter`,
`slice`, `cast` and `mask` build lazy wrappers only. `compare` needs
`all_valid` to decide whether an uncoercible constant can be answered
without null checks, and `is_constant` reads its children, so both
legitimately take a context.

`take_pushes_down_without_executing` pins the new behavior: it asserts
that `take` on a wide array reduces to the encoding rather than being
left as a `vortex.dict`, and fails with "got vortex.dict" if the rule is
unregistered.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

The wide `DecimalByteParts` columns were added to the existing
`decimal_byte_parts.vortex` fixture, which breaks the compat contract.
`DESIGN.md` states it directly under "Fixture evolution": a fixture's
`build()` is immutable once published, because `check` compares files
written by older releases against what `build()` produces today. Adding a
column changes the schema the generator emits, so the check fails against
every previously published version — exactly the regression the fixture
exists to catch, reported against unrelated releases.

`decimal_byte_parts.vortex` is restored to its published definition, and
the wide cases move to a new `decimal_byte_parts_wide.vortex` with a
comment recording why the split exists rather than leaving the next person
to rediscover the rule. The new fixture gains a negative `i128` column so
sign extension above the MSP is exercised on read back, alongside the
one-lower-part and nullable three-lower-part cases.

Verified with `generate` followed by `check --mode exact`: 36 fixtures
pass, and `decimal_byte_parts.rs` is byte-identical to its pre-branch
state.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

`i256::from_parts` takes a `u128` and an `i128`, so each row of the
assembly loop ends in `u128::from(w0) | (u128::from(w1) << 64)`. The
reasonable suspicion is that this is worse than storing four `u64`s by
hand, since 128-bit integers have a reputation for lowering badly.

`i256_row_words` is that hand-written version: it builds a `u64` lane
buffer and reinterprets it as `i256` at the end, so no 128-bit value is
ever written. Over 65,536 rows it ties the shipped shape across four runs
(`fastest` 224-228 µs against 227-236 µs), which is inside the noise on
this host.

Disassembly explains the tie and is the more durable evidence. Neither
shape emits a single `shld`/`shrd`, and both compile to four plain 64-bit
stores per row at offsets 0x0/0x8/0x10/0x18. The `i128` loop is the same:
`(i128::from(msp) << 64) | i128::from(part)` becomes two 64-bit stores
with no shift at all. A shift by a constant multiple of 64 followed by an
or is pure data movement and LLVM recognizes it as such; the 128-bit
codegen actually worth avoiding is division and remainder, which call into
compiler-rt, and shifts by a runtime amount. Neither appears in this code.

So no change to the assembly loops. The variant and the reasoning stay in
the benchmark, because "avoid the u128" is a rewrite someone will propose
again and this is the answer.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

The hand-written shape variants have served their purpose: the design
questions they were written to answer are settled, and the answers are
recorded in the module docs. Keeping them means maintaining a second copy
of the assembly loop that no test covers and that silently stops
representing the shipped code the moment that loop changes.

`canonicalize_byte_parts` stays. It goes through the array API rather than
duplicating the loop, so it tracks whatever shape the crate ships and
works as a regression guard. The module docs keep the measured conclusions
— const part count is 1.85x on `i256`, the pre-sized write is 1.6x on
`i128`, columnar loses on both, and hand-written 64-bit words tie the
`u128` packing because neither emits a shift — with a note that the
variants are recoverable from history if a future change needs to re-run
the comparison rather than trust the numbers.

346 lines to 105.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

An `i256` is exactly `{_0: u64, _1: u64, _2: u64, _3: i64}`: three
unsigned words beneath a single signed one. That is the same shape this
encoding stores — unsigned lower parts under a signed most significant
part — and it is why splitting and reassembling are pure reinterpretation
rather than arithmetic. No carry crosses a word boundary, so each word
compresses independently and goes back verbatim.

The code did not say so. Three sites open-coded the same word math with
`to_parts`/`from_parts` and shifts: `split_i256` unpacking, and
`combine_i256` and `assemble_i256` packing, the latter two character for
character identical. A reader had to re-derive the layout at each one, and
`split_i256` carried a `cast_possible_truncation`/`cast_sign_loss` expect
that hid where the truncation was meant to happen.

`i256_to_words` and `i256_from_words` now name the reinterpretation, and
`sign_extended_words` names the other half of the invariant: the words
above the most significant part are its sign. `split_i256` reads as the
inverse of `assemble_i256` at `K == MAX_LOWER_PARTS`, and says so.

Codegen is unchanged. `assemble_i256` still compiles to four plain 64-bit
stores per row at offsets 0x0/0x8/0x10/0x18 with no `shld`/`shrd`, and the
only shifts in the function are index scaling and a single `sar $0x3f` —
the branchless sign broadcast, which is the ideal lowering of the fill.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

A reader that predates lower parts expects this encoding to have exactly
one child, so a file containing a multi-child `DecimalByteParts` is one it
cannot open. Introducing lower parts is now gated behind
`unstable_encodings` at both places that can introduce them.

`DecimalByteParts::try_new_with_lower_parts` rejects a non-empty
lower-parts list without the feature, and names the feature in the error.
`try_new` builds a single child and is unaffected. In the compressor, the
decimal scheme leaves values too wide for one signed part as the canonical
decimal instead of splitting them, and reports `num_children` as 1 —
restoring exactly the pre-lower-parts behaviour, which was to return the
narrowed array uncompressed.

The gate is on *introducing* lower parts, not on having them. Rebuilding
an array whose parts already exist goes through a new crate-private
`rebuild_with_lower_parts`, which every compute kernel uses via
`map_parts`/`with_msp`, and `deserialize` is untouched. Gating those too
would mean a build without the feature could not read a file written by a
build with it — strictly worse than not being able to write one. An
earlier revision of this change did gate them, and
`compute_over_existing_lower_parts_is_not_gated` fails without the split:
reverting `map_parts` to the public constructor breaks filter, take,
slice and the consistency suite on every wide array.

That test also drove the gate's shape. Letting the crate's own unit tests
through the gate via `cfg!(test)` would have hidden exactly that bug,
since unit tests would no longer run the configuration they ship. The gate
is therefore purely `cfg!(feature = ...)`, and the test helpers that build
wide arrays call `rebuild_with_lower_parts` explicitly, so a default
`cargo test` still covers the multi-part paths while running the same gate
production does. `tests/lower_parts_gate.rs` covers the gate itself.

Tests that assert lower parts are *produced* — the btrblocks split and
compression-ratio tests, and the vortex-file round trip — are gated on the
feature, since without it the compressor deliberately declines. The
benchmark declares `required-features` for the same reason.

The wide compat fixture is gated too: it is a written file, so generating
it by default would emit precisely what the gate exists to prevent. A
default `generate` produces 35 fixtures, and 36 with the feature; `check
--mode exact` passes in both.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Gating construction and the compressor was not enough. An array read from
a file can be handed straight back to a writer without passing through
either: `deserialize` is deliberately ungated so a build without the
feature can still read such files, and the write allow-list checks only
the encoding id, not how many children it carries — `ALLOWED_ENCODINGS`
inserts `DecimalByteParts.id()` unconditionally. A build that could never
have constructed a multi-child array could therefore still emit one.

This was demonstrable, not theoretical: `test_serde_round_trip` with three
lower parts passed on default features before this change.

`VTable::serialize` now refuses an array carrying lower parts unless the
feature is on. That is the last point before bytes reach a file, so it
covers the pass-through path as well as anything else that reaches the
writer. Reading stays untouched, and so does compute over an array that
already has lower parts.

`serializing_read_lower_parts_is_gated` pins it, going through
`deserialize` to obtain the array exactly as opening a file would, and
asserting the write is refused with an error naming the feature. The three
wide `test_serde_round_trip` cases move to a feature-gated variant, since
without the feature serializing them is now the refusal being tested.

Note this makes a stable build unable to rewrite a wide array it just
read, so copying or compacting such a file fails loudly rather than
producing something old readers cannot open. That is the intended
trade-off while the format is unstable, but it is a behaviour change for
read-modify-write on files written with the feature.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Uses `hegeltest`, the Hypothesis-based property testing crate that
`spiraldb/fastlanes` already depends on, so the two repositories share a
generator vocabulary and shrinking behaviour.

Every property has the same shape: the encoding must be indistinguishable
from doing the same thing to the canonical `DecimalArray`. Seven of them
cover split/assemble round tripping, a serialize/decode round trip through
the file path, per-row `scalar_at` against bulk canonicalization, and
filter, slice and take.

The properties were checked against deliberate mutations rather than
assumed to be load bearing. Six real mutations, all caught: reversing the
lower-part order in assembly, placing the MSP one word too low, swapping
the `i256` word-pair packing, dropping the lower part on the `i128` path,
reversing the order in `split_i256`, and dropping the sign fill. A seventh
— logical instead of arithmetic shift in `split_i128` — is an equivalent
mutant, since truncating to `i64` makes the shift kind unobservable, and
is correctly not flagged.

Dropping the sign fill initially survived, which is why
`msp_below_the_top_word_sign_extends` exists. `split_decimal` always emits
three lower parts for an `i256`, and at three parts every word is written,
so the fill is dead on that path — the round-trip properties structurally
cannot reach it. Only a directly constructed array with a most significant
part below the top word does. That property computes its expectation
independently of the assembly loop: with two lower parts the MSP occupies
bits 191..128, exactly the low half of an `i256`'s signed `i128` half, so
`i128::from` performs the sign extension the encoding is supposed to.

The test target declares `required-features = ["unstable_encodings"]`,
since building multi-part arrays is what the write gate restricts.

Running the properties writes an example database to `.hegel/`, which is
generated state rather than source — the same role `.hypothesis/` plays for
Python, and ignored alongside it.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Three changes that move the lower-parts restriction to where it belongs.

The compressor now only handles decimals that fit a single signed part.
Anything still wider than 64 bits after narrowing is left as the canonical
decimal, unconditionally — this is a property of the width, not of a
feature, so `num_children` is back to 1 and the lower-part compression loop
is gone. That restores the pre-lower-parts behaviour permanently rather
than behind a flag.

`serialize` keeps refusing an array carrying lower parts without
`unstable_encodings`, and is now the only gate. It is the right place for
it: the write allow-list checks the encoding id rather than the child
count, so this is the single point every path to a file passes through,
whether the array came from a constructor, a compute kernel, hand-assembled
slots, or a previous read.

The constructor limb check is removed. Building lower parts in memory is
allowed again, which is what reading a file needs anyway, so
`rebuild_with_lower_parts` — which existed only to bypass that check —
disappears with it; `map_parts`, `with_msp` and the test helpers go back to
the public constructor. Everything else `validate` enforces is untouched:
signedness, dtypes, lengths, the part-count bound, and the width-against-
precision check all still apply.

Because the compressor no longer splits, the btrblocks tests asserting one
and three lower parts are replaced by one asserting wide values are left
canonical, and the compression-ratio test is dropped — it measured the win
from splitting, which no longer happens. The vortex-file round trip loses
its ratio assertion for the same reason but is now ungated, so wide
decimals are covered by a default `cargo test` instead of only under the
feature. The property tests and the benchmark no longer need the feature
either, since neither serializes.

`unstable_encodings` now reaches the encoding through `vortex` and
`vortex-file` rather than `vortex-btrblocks`, which no longer has an
opinion about lower parts. Verified end to end: a default `generate`
produces 35 fixtures, and 36 with the feature, `check --mode exact` passing
in both.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

…e-parts-pr-p0ugog

Merging develop brought `trace_tests`, which snapshots the optimizer trace
for a compressed lineitem scan. Two entries move, both because of changes
in this branch, and both are the intended behaviour rather than a drift in
output.

The filter trace names `FilterReduceAdaptor(DecimalByteParts)` where it
named `DecimalBytePartsFilterPushDownRule`. That rule was byte-for-byte
what the adaptor already did and was registered ahead of it, so removing it
leaves the adaptor to do the same work under its own name. Parent, child
and result in the trace line are unchanged.

The take trace gains three lines: the decimal column now reduces during
optimization rather than being deferred to execution, which is exactly what
moving `take` from `TakeExecute` to `TakeReduce` was for. The trace is the
first end-to-end evidence of that change — the encoding's own tests can
only observe it one array at a time.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>